Mobile brain-based device having a simulated nervous system based on the hippocampus

ABSTRACT

A brain-based device (BBD) having a physical mobile device NOMAD controlling and under control by a simulated nervous system. The simulated nervous system is based on an intricate anatomy and physiology of the hippocampus and its surrounding neuronal regions including the cortex. The BBD integrates spatial signals from numerous objects in time and provides flexible navigation solutions to aid in the exploration of unknown environments. As NOMAD navigates in its real world environment, the hippocampus of the simulated nervous system organizes multi-modal input information received from sensors on NOMAD over timescales and uses this organization for the development of spatial and episodic memories necessary for navigation.

PRIORITY CLAIM

This application is a continuation of U.S. patent application Ser. No.12/331,133, entitled “Mobile Brain-Based Device Having a SimulatedNervous System Based on the Hippocampus,” by Gerald M. Edelman et al.,filed Dec. 9, 2008, which is a continuation of U.S. patent applicationSer. No. 11/179,371, entitled “Mobile Brain-Based Device Having aSimulated Nervous System Based on the Hippocampus,” by Gerald M. Edelmanet al., filed Jul. 12, 2005, now U.S. Pat. No. 7,467,115, issued Dec.16, 2008, which claims priority under 35 U.S.C. 119(e) to U.S.Provisional Patent Application No. 60/588,107, filed Jul. 15, 2004,entitled “Mobile Brain-Based Device Having a Simulated Nervous SystemBased on the Hippocampus,” by Gerald M. Edelman et al., whichapplications are incorporated herein by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT

This invention was made with Government support under N00014-03-1-0980awarded by the Office of Naval Research. The United States Governmenthas certain rights in the invention.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the United States Patent andTrademark Office patent file or records, but otherwise reserves allcopyright rights whatsoever.

FIELD OF THE INVENTION

The present invention relates to brain-based devices and, moreparticularly, to brain-based devices which can navigate in a real worldenvironment.

BACKGROUND OF THE INVENTION

Intelligent systems have been developed which are intended to behaveautonomously, automate tasks in an intelligent manner, and extend humanknowledge. These systems are designed and modeled based on essentiallythree distinct fields of technology known, respectively, as

(1) artificial intelligence (AI);

(2) artificial neural networks (ANNs); and

(3) brain-based devices (BBDs).

The intelligent systems based on AI and ANN include digital computerswhich are programmed to perform tasks as far ranging as playing chess torobotics. AI algorithms are logic-based and preprogrammed to carry outcomplex algorithms implemented with detailed software instructions. ANNsare an oversimplified abstraction of biological neurons that do not takeinto consideration nervous system structure (i.e. neuroanatomy) andoften require a supervisory or teacher signal to get desired results.BBDs, on the other hand, are based on different principles and adifferent approach to the development of intelligent systems.

BBDs are based on fundamental neurobiological principles and are modeledafter the brain bases of perception and learning found in living beings.BBDs incorporate a simulated brain or nervous system with detailedneuroanatomy and neural dynamics that control behavior and shape memory.BBDs also have a physical instantiation, called a morphology orphenotype, which allows active sensing and autonomous movement in theenvironment. BBDs, similar to living beings, organize unlabeled signalsthey receive from the environment into categories. When a significantenvironmental event occurs, BBDs, which have a simulated neuronal areacalled a value system, adapt the device's behavior.

The different principles upon which logic-based intelligent systems andBBDs operate are significant. As powerful as they are, logic-basedmachines do not effectively cope with novel situations nor process largedata sets simultaneously. By their nature, novel situations cannot beprogrammed beforehand because these typically consist of unexpected andvarying numbers of components and contingencies. Furthermore, situationswith broad parameters and changing contexts can lead to substantialdifficulties in programming. And, many algorithms have poor scalingproperties, meaning the time required to run them increasesexponentially as the number of input variables grows.

A challenging problem in intelligent systems such as autonomous roboticsystems, therefore, is the successful exploration of unknown terrain.Exploration in the real world requires navigation and spatial memorytasks to be solved. However, the memory required to be successful inthis task requires features only found in living beings and that arebelieved to be the hallmark of “episodic” memory, i.e. (1) the abilityto put together multi-modal sensory information into coherent patterns,(2) the ability to put together information over time and recalltemporal sequences, and (3) the ability to use memory for goal directedbehavior. The hippocampus, which is located in the medial temporal lobeof the brain, and which has been well studied clinically andphysiologically, is known to be crucial for memory and navigation inhumans and animals. Consequently, the hippocampus has inspired priorbiologically based navigation systems, some of which are computationalhippocampal models and others of which are hippocampal models that havebeen applied to robots, but both of which have their limitations.

Prior computational hippocampal models have been run as simulations on acomputer with virtual inputs. These computational hippocampal modelsmake assumptions and use “a priori” information in order to get theappropriate responses to the inputs. For example, the hippocampal“place” cells (i.e. neurons that are active when the animal is in aspecific location of the environment) of these computational modelsrespond to a sensory input combination specifically engineered by themodeler, such as a 2-D (two-dimensional) point in Cartesian space. Partof the reason for these assumptions having been made is due to thecomputational hippocampal model not being situated in a realenvironment, thereby necessitating these biases.

Some of the computational hippocampal models have investigated theinteraction between the hippocampus and other areas of the brain, suchas the neocortex. However, in some the anatomy of these models was verysimple and did not truly reflect hippocampal-cortical interactions in ameaningful way. One such model integrates the hippocampal formation withvisual and path integration processing that could be thought of ascortical inputs and does make an assumption that path integration issolved by a moving bump of activity that reflects the animal movement ona map of the environment. This would not be feasible if the animal wasin a real-world environment. Others have constructed a sophisticatedmodel of the hippocampus with the appropriate connections in thehippocampus proper, with this model having been used to investigatememory conditions and issues. Although this model is quite detailed, theinputs into it are tokens or symbols which have no bearing to theprocessed multi-modal sensory input that converges on the hippocampus.This model produces an abstract output pattern that is read out as amemory recall. However, it is hard to resolve this response with thatof, for example, a rodent where hippocampal responses lead to actualadaptive behavior.

While hippocampus models also have been instantiated on mobile robots,many of these also make assumptions, such as the “a priori” informationdriving the response of hippocampal “place” cells or of a map that isinput to the hippocampus. A few robotics models, which do include aneural simulation controlling the mobile robot in a navigation task,learn the mappings and hippocampal responses by autonomous exploration.One such model was very loosely tied to neurobiology and used learningalgorithms similar to what is known as back propagation for learning.This developed a Simultaneous Localization and Mapping algorithminspired by the rodent hippocampus, called RatSLAM, which is a hybridbetween Artificial Intelligence SLAM systems and attractor dynamicsthought to be represented in the hippocampus to create map-likerepresentations of the environment. Yet others constructed a roboticsmodel that integrated visual input with a head direction system, inwhich “place” cells, developed in the hippocampal layer of the modelduring exploration, and a biologically-based reward system drovelearning between the “place” cells and goal-directed behavior. However,some features of the model are not true to the biology. (1) First, whenthe robotic model decided that a new place had been discovered, a“place” cell was added to a growing hippocampal layer. In a real beingsuch as a rodent, a hippocampal cell can respond to multiple placesdepending on the context or any combination of inputs. This flexibilitymakes the hippocampus a multi-purpose memory map as opposed to aspecialized positioning system. Also, hippocampal cells are not added onan as needed basis. (2) Second, the robotic model was feedforward anddid not take into consideration the intrinsic and extrinsic looping thatis a feature of the hippocampus. In yet another system, there was builta hippocampal neuroanatomy and a biologically-based goal system, whichwas tested on a mobile robot. However, “place” cells were artificial inthe sense that the responses were designed to uniformly cover a grid ofa controlled environment. The reward learning was used to build acognitive map between these places. Moreover, although much of thedetails found in the hippocampus and the surrounding areas were includedin the model, information flowed in a purely feedforward fashion throughthe model and did not loop back through the entorhinal cortex and thenon to the neocortex.

Over ten years ago, a statistical framework for simultaneously creatingmaps while localizing the robot's position was developed, which has beencommonly referred to as SLAM (Simultaneous Localization and Mapping).Since that time, the field of robotic mapping has been dominated byprobabilistic techniques. The most popular is the estimation theoreticor Kalman filter based approach because it directly provides both arecursive solution to the navigation problem and a means of computingconsistent estimates for vehicle and landmark locations based onstatistical models of vehicle motion and landmark observations. Theserobotic approaches typically measure the distance to landmarks by laserrange finders, sonar, or radar to create a map of landmarks andsimultaneously estimate the position of the robot. These techniques havebeen very successful in creating maps for robots in certain officeenvironments, in outdoor environments, and for unmanned aerial vehicles.However, these techniques have not addressed the problem of recognizingobjects or situations and taking the appropriate actions, i.e.navigating.

SUMMARY OF THE INVENTION

The present invention is a brain-based device (BBD) that is able toadapt to varying terrain by learning which aspects of the terrain it cantraverse and which aspects it must circumvent. The BBD makes decisionson aspects of the environment that go beyond mapmaking of prior roboticsystems. These decisions require perceptual categorization of localobject features, which the BBD of the present invention makes, therebyenabling the BBD to plot a course.

The BBD of the present invention includes a simulated nervous systemhaving neural areas, specifically the hippocampus, the parietal cortex,the inferotemporal cortex, and a thalamic head direction system. Thebi-directional connectivity between the hippocampus and these regions(see FIG. 3A), as well as the intricate connectivity within thehippocampus (see FIG. 3B), comprise a system that loops multimodalinformation over time and allows the formation of associative memorieswith a temporal context. The BBD also has a mobility base enabling it tomove about in a real world environment under control of the simulatednervous system, together with a multi-sensor input enabling the BBD torecognize objects and select an action that has value consequences.

For those BBDs of the present invention that are to navigate in acomplex terrain, these may combine signals from a laser or radar-basedSimulated Localization and Mapping (SLAM) with the BBDs objectrecognition system and map of actions in space.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a pictorial view of a physical, mobile brain-based device.

FIG. 2A is a schematic of the enclosed environment in which the BBD ofthe present invention moves.

FIG. 2B is a snapshot photograph of the BBD in the enclosed environmentdepicted in FIG. 2A.

FIG. 3A is a schematic high level diagram of the regional and functionalneuroanatomy of the simulated nervous system of the BBD of the presentshowing the neural simulation connection.

FIG. 3B shows in detail the connectivity within the hippocampus regionof FIG. 3A. The inset depicts the synaptic change rule, which is basedon the BCM learning rule.

FIG. 4 pictorially shows representative trajectories of the BBD of thepresent invention during hidden platform training trials.

FIG. 5 is a graph of mean escape times of several embodiments of the BBDof the present invention tested during a hidden platform task.

FIGS. 6A and 6B show pictorially the trajectories of respectiveembodiments of the BBD of the present invention during a probe trial.

FIGS. 7A-7D show four representation “place” cells recorded in asimulated neuronal area CA1.

FIGS. 8A-8F are charts depicting journey dependent and independent cellsin the hippocampus model.

FIGS. 9A-9H illustrate a representative journey dependent “place” cellin neuronal area CA1 during respective trials.

FIG. 10 is an exemplary illustration of a system in accordance withvarious embodiments of the present invention.

FIG. 11 is a flow diagram illustration of neural simulatorinitialization.

FIG. 12 is a flow diagram illustration of the master component inaccordance with various embodiments of the invention.

FIG. 13 is a flow diagram of a neural simulator in accordance withvarious embodiments of the invention.

DETAILED DESCRIPTION

Aspects of the invention are illustrated by way of example and not byway of limitation in the figures of the accompanying drawings in whichlike references indicate similar elements. It should be noted thatreferences to “an”, “one” and “various” embodiments in this disclosureare not necessarily to the same embodiment, and such references mean atleast one. In the following description, numerous specific details areset forth to provide a thorough description of the invention. However,it will be apparent to one skilled in the art that the invention may bepracticed without these specific details. In other instances, well-knownfeatures have not been described in detail so as not to obscure theinvention.

FIG. 1 is a pictorial view of a brain-based device (BBD) of the presentinvention which includes a physically instantiated device, shown as oneexample as a mobile Neurally Organized Mobile Adaptive Device (NOMAD) 10which can explore its environment and develop adaptive behavior whileexperiencing the environment. The brain-based device BBD also includes asimulated nervous system 12 (FIGS. 3A-3B) for guiding NOMAD 10 in itsreal-world environment. In one embodiment, the simulated nervous system12, as will be further described, can run on a cluster of computerworkstations (see FIG. 10) remote from NOMAD 10. In this embodiment,NOMAD 10 and the computer workstations communicate with one another viawireless communication, thereby enabling untethered exploration of NOMAD10.

In the physical device example, NOMAD 10, as shown in FIG. 1, has amobile base 14 and a CCD camera 16 for vision. NOMAD 10 also hasodometry 18 for self-movement cues, effectors 20, seven infrared (IR)detectors 22 encircling NOMAD 10 for obstacle avoidance, and onedownward facing IR detector 24 to detect a hidden platform (see FIGS.2A-2B) within the enclosed environment in which it moves. On the top ofNOMAD 10 are LEDs 26, which are detectable by two cameras (describedbelow) situated over the environment in which NOMAD 10 moves, to trackNOMAD's position in the environment.

The simulated nervous system 12 guides the behavior of NOMAD 10 based onthe organization of real anatomy and physiology, and emphasizing aliving organism's interaction with the environment. Thus, this behavioris based on the following design principles: (1) NOMAD 10 should engagein a behavioral task; (2) NOMAD's behavior should be controlled by asimulated nervous system having a design that reflects the brain'sarchitecture and dynamics; (3) NOMAD 10 should be situated in thereal-world; and (4) the behavior of NOMAD 10 and the activity of thesimulated nervous system 12 should allow comparisons with empiricaldata. With these characteristics, BBD simulations tend to requirenetworks of neuronal elements that reflect vertebrate brain architectureand dynamics, high performance computing to run the network inreal-time, and the engineering of physical devices, e.g. NOMAD 10, toembody the network, all of which is described below.

FIG. 2A shows schematically a layout of the enclosed environment 28 inwhich NOMAD 10 moves. The enclosed environment 28 is, for example,16′×14′ with black walls 30 and black flooring 32. Sets of coloredconstruction paper 34 of varying colors are hung on each of the blackwalls 30. As shown, each set 34 has a plurality of pieces ofconstruction paper, with each piece in a given set 34 being of the samewidth but different in width than pieces in the other sets 34.

FIG. 2A also shows a hidden platform 36 which is 24″ in diameter andmade of reflective black construction paper. The hidden platform 36 isplaced in the center of the upper right quadrant of the enclosedenvironment 28 for “hidden platform tasks” or trials described below.NOMAD 10 is able to detect the hidden platform 36, by use of thedownward facing IR detector 24, when it is positioned over the platform36.

FIG. 2B is a snapshot showing NOMAD 10 in the enclosed environment 28.NOMAD 10 is also shown performing the hidden platform task which is usedfor assessing the BBD's spatial and episodic memory. As will be furtherdescribed, the hidden platform task includes two phases, a set of“training” trials and a “probe” trial. In the training phase, NOMAD 10will begin a number of trials, starting its movement from any one of anumber of positions on the floor 32 until it encounters the hiddenplatform 36 or it times out, e.g., after 1000 seconds or 5000 cycles.After training, a probe trial is used to assess the BBD's memoryperformance. In the probe trial, the hidden platform 36 is removed fromthe enclosure 28 and NOMAD 10 is allowed to move about the enclosure for5000 cycles.

FIG. 3A is a schematic of the regional and functional neuroanatomy ofthe simulated nervous system 12, which includes the intricate anatomyand physiology of the hippocampus and its surrounding regions. Thissimulated nervous system 12 can integrate spatial signals from numerousobjects in time and provide flexible navigation solutions to aid in theexploration of unknown environments, such as the environment 28. Bysimultaneously sampling from its neural regions during a navigationtask, the architecture of the hippocampus, as described more fullybelow, provides a means to organize multi-modal information overdifferent timescales which is important for the development of spatialand episodic memories.

The simulated nervous system 12, as shown in FIG. 3A, is modeled on theanatomy and physiology of the mammalian nervous system but, as can beappreciated, with far fewer neurons and a much less complexarchitecture. Simulated nervous system 12 includes a number of neuralareas labeled according to the analogous cortical and subcorticalregions of the human brain. Thus, FIG. 3A shows respective neural areaslabeled as V1 Color, V2/4 Color, IT, V1 Width, V2/4 Width, Pr, HD, ATN,BF, Hippocampus, R+, S, R−, and M_(HDG). Each neural area V1 Color, V2/4Color, etc. contains different types of neuronal units, each of whichrepresents a local population of neurons. Each ellipse shown in FIG. 3Adenotes a different neural area, with each such area having manyneuronal units. To distinguish modeled or simulated neural areas fromcorresponding regions in the mammalian nervous system, the simulatedareas are indicated in italics, e.g. IT.

The neuroanatomy of FIG. 3A also shows schematically the neuralsimulation 12 connectivity via various projections P (see arrows in theFigure). A projection P contains multiple synaptic connections betweenneuronal units. A synaptic connection allows a signal to be sent from apre-synaptic neuronal unit to a post-synaptic neuronal unit. ProjectionsP can either be within a neural area or between neural areas.Furthermore, projections P have properties as indicated by the legend inFIG. 3A, which are (1) “voltage independent”, (2) “voltage dependent”,(3) “plastic”, (4) “inhibitory,” and (5) “value dependent” and will bedescribed in detail below.

Input to the simulated neural system 12 comes from the CCD camera 16,wheel odometry 18, and IR sensors 22, 24 for detection of walls 30 andhidden platform 36 of environment 28. The neural areas of simulatedneural system 12 are analogous to the visual cortex (V1, V2/4), theinferotemporal cortex (IT), parietal cortex (Pr), head direction cells(HD), anterior thalamic nuclei (ATN), motor areas for egocentric heading(M_(HDG)), a value system (S), and positive and negative reward areas(R+, R−). The hippocampus is connected with the three major sensor inputstreams (IT, Pr, ATN), the motor system (M_(HDG)), and the value system(S). For clarity, the intrinsic connections within each of the neuralareas are omitted from FIG. 3A.

FIG. 3B shows the detailed connectivity within the hippocampal region.The modeled hippocampus contains areas analogous to entorhinal cortex(ECIN, ECOUT), dentate gyms (DG), the CA3 subfield, and the CA1subfield. These areas contain interneurons that cause feedbackinhibition (e.g. CA3→CA3FB→CA3), feedforward inhibition (e.g.DG→CA3FF→CA3), and rhythmic inhibition (e.g. BF→Hippocampus). (FIG. 3A).

Much more detail of the simulated nervous system 12 is given in Table 1and Table 2 described below. But overall, in the version of thesimulated nervous system 12 used in the training and probe trialsdescribed in detail below, there are a total of 50 neuronal areas,90,000 neuronal units within the 50 neuronal areas, and approximately1.4 million synaptic connections.

The simulated nervous system 12 shown in FIG. 3A is comprised of fivesystems: (1) a visual system 38, (2) a head direction system 40, (3) ahippocampus formation 42, (4) a value system 44, and (5) an actionselection system 46.

FIG. 3A. Visual System 38.

The visual system 38 is modeled on the primate occipitotemporal orventral cortical pathway and a dorsal cortical pathway. The ventralcortical pathway shown in FIG. 3A, in the visual system(V1_color→V2/4_color→IT), contains neuronal units in successive areashaving progressively larger receptive fields until, in inferotemporalcortex (IT), receptive fields cover the entire visual field and have notopography. The dorsal cortical pathway shown in FIG. 3A(V1_width→V2/4_width→Pr) contain neuronal units that respond to the sizeand position of objects.

Visual images from the CCD camera 16 are filtered for color and edgesand the filtered output directly affects neural activity in areaV1-Color and V1-Width. The CCD camera 16 sends, for example, 320×240pixel RGB video images, via an RF transmitter on NOMAD 10, to a framegrabber attached to one of the computer workstations (described below)running the neural simulation. The image is spatially averaged toproduce an 80×60 pixel image. Different sized Gabor filters (8×8, 16×16,32×32, and 64×64) may be used to detect vertical edges of varyingwidths. The output of the Gabor function maps directly onto the neuronalunits of the corresponding V1 width sub-areas (V1-width8, V1-width16,V1-width32, and V1-width64) (not shown in FIG. 3A). Color filters (e.g.,red positive center with a green negative surround, red negative centerwith a green positive surround, blue positive with red-green negative,and blue negative with red-green positive) may be applied to the image.The outputs of the color filters are mapped directly onto the neuronalunits of V1 Color sub-areas V1-red, V1-green, V1-blue, and V1-yellow(not shown in FIG. 3A). V1 neuronal units project retinotopically toneuronal units in V2/V4.

FIG. 3A. Head Direction System 40.

Neurons in the areas HD are often called “head direction” cells.Information obtained from the wheels 20 of NOMAD 10 is used to estimatethe current heading of NOMAD 10. This information is input into the headdirection neural area (HD). Each of the 360 HD neuronal units (seeTable 1) has a cosine tuning curve, which responds maximally to apreferred heading with a tuning width of π radians:cos(HD_(i)−curr_heading)⁵;where HD_(i) is a head direction cell with a preferred direction of

$( {\frac{i}{360}2\pi} ),$i ranges from 0 to 359, and curr_heading is NOMAD's heading, which iscalculated from odometer information.

The head direction cells project topographically to an area analogous tothe anterior thalamic nucleus (see HD→ATN in Table 2 and FIG. 3A) and toa motor area based on heading (see HD→M_(HDG) in Table 2 and FIG.3A—note, this path not in FIG. 3A). A new heading for NOMAD 10 is chosenbased on the activity in neuronal area M_(HDG) (see FIG. 3A—ActionSelection, below, for further details).

Hippocampal Formation—Neural Area Hippocampus 42.

The architecture of the simulated hippocampal formation is based onrodent neuroanatomy. The input streams into the hippocampus are from theassociative cortical areas in the simulated nervous system and arrive atthe hippocampus via the entorhinal cortex (see ATN→EC_(IN), IT→EC_(IN),Pr→EC_(IN) in Table 2 and FIGS. 3A-3B). The perforant path projectsmainly from entorhinal cortex to the dentate gyms but also to the CA3and CA1 subfields (see EC_(IN)→DG EC_(IN)→CA3, EC_(IN)→CA1 in Table 2and FIG. 3B). The mossy fibers (see DG→CA3 in Table 2 and FIG. 3B),Schaffer collaterals (see CA3→CA1 in Table 2 and FIG. 3B), and divergentprojections from the hippocampus back to the cortex (seeCA1→EC_(OUT)→ATN,IT,Pr) in Table 2 and FIGS. 3A-3B) are also present inthe simulated nervous system 12. The prevalent recurrent connectivityfound in the hippocampal formation is also included in the simulatednervous system 12 (see EC_(IN)→EC_(OUT), DG→DG, and CA3→CA3 in Table 2and FIG. 3B). All of the connections within the hippocampus proper(i.e., EC_(IN), EC_(OUT), DG, CA3, CA1) are value-independent andplastic (see section Synaptic Plasticity, below).

Unique patterns of intrinsic and extrinsic, feedback and feedforwardinhibitory connections in the hippocampus may play an important role inhippocampus processing. Consequently, the simulated nervous system 12includes feedback inhibitory connections (see EC→EC_(FB)→EC,DG=DG_(FB)→DG, CA3→CA3_(FB)CA3, CA1→CA1_(FB)→CA1 in Table 2 and FIG. 3B)and feedforward inhibitory connections (see DG→CA3_(FF)→CA3,CA3→CA1_(FF)→CA1 in Table 2 and FIG. 3B). These connections are usefulfor separating inputs and network stability.

Basal Forebrain and Theta Rhythm (Table 2).

The simulated basal forebrain (BF) provides an extrinsic theta rhythmfor the neural simulation. The function of the simulated basal forebrainarea is to gate input into the hippocampus and keep activity levelsstable. The BF area has a rhythmic activity over 13 simulation cycles:BF(t)=theta(t mod 13);where theta={0.01, 0.165, 0.33, 0.495, 0.66, 0.825, 1.00, 0.825, 0.66,0.495, 0.33, 0.165, 0.01}. BF projects to all hippocampal areas withinhibitory connections (see BF→EC_(IN),EC_(OUT),DG,CA3,CA1 in Table 2).The level of inhibition, which is adaptive, keeps the activity inhippocampal regions within specific ranges:Δsf _(r)(t)=(s _(r)(t)−tgt _(r))BF_(r)(t)=BF(t)+sf _(r)(t);where r denotes the region (i.e. EC_(IN), EC_(OUT), DG, CA3, CA1),sf_(r)(t) is the scale factor at time t, s_(r)(t) is the percentage ofactive neuronal units in region r at time t, tgt_(r) is the desiredpercentage of active units in area r (EC_(IN)=10%, EC_(OUT)=10%, DG=20%,CA3=5%, and CA1=10%), and BF_(r)(t) is the pre-synaptic neuronal unitactivity for a BF to hippocampus region r connection.FIG. 3A. Value Systems and Temporal Difference Learning 44.

Activity in the simulated value systems 44 signals the occurrence ofsalient sensory events experienced by NOMAD 10 and this activitycontributes to the modulation of connection strengths in the pathwaysshown. Initially, value system S is activated by the IR detector 24 thatdetects hidden platform 36 (see R⁺→S in Table 2 and FIG. 3A), causingpotentiation of value dependent connections (CA1→S and CA1→M_(HDG)), orby obstacle avoidance the IR detectors 22 (see R⁻→S in Table 2 and FIG.3A), causing depression of value dependent connections. After learning,pursuant to the training trials mentioned above and described more fullybelow, the area CA1 can have an effect on area S activity. The magnitudeof potentiation or depression is based on a neural implementation of atemporal difference (TD) learning rule:

${{TD}(t)} = \{ {\begin{matrix}{{{R^{+}(t)} - \overset{\_}{S( {t - \tau} )}};} & {R^{+} > 0} \\{{\overset{\_}{S( {t - \tau} )} - {R^{-}(t)}};} & {R^{-} > 0} \\{{\overset{\_}{S(t)} - \overset{\_}{S( {t - \tau} )}};} & {otherwise}\end{matrix};} $where S(t) is the average activity of the value system S at time t, τ isone theta cycle or 13 simulation cycles, R⁺ is a positive reward andequal to 1 if the downward facing IR detector 24 is triggered, meaningthat NOMAD 10 is over the hidden platform 36, and R is a penalty andequal to 1 if one of the seven IR detectors 22 around NOMAD's base istriggered, meaning that NOMAD 10 is too close to a wall 30. The basicidea of the temporal difference TD rule is that the learning is based onthe difference between temporally successive predictions of rewards. Inother words, the goal of the learning is to make the learner's currentprediction for the current input pattern more closely match the nextprediction at the next time interval (τ). If the predicted valueincreases, TD is positive and affected synaptic connections of thesimulated nervous system 12 are potentiated, and if the predicted valuedecreases TD is negative and affected synaptic connections aredepressed. After some experience, values should increase as NOMAD 10heads toward the platform 36 and thus reinforce movements towards theplatform 36. Alternatively, values should decrease as NOMAD 10 nearsobstacles, and thus reinforce movements away from walls 30 and otherobstacles. Further details on how the temporal difference is applied toindividual synaptic connections are given below when describing SynapticPlasticity.FIG. 3A. Action Selection and Exploratory Behavior 46.

NOMAD 10 moves forward for 3 theta cycles (39 simulation cycles) andthen selects a new heading, as described below. If NOMAD 10 detects anobstacle, it may reverse its direction for 24 inches, and then turn awayfrom the IR sensor 22 that detected the obstacle. If NOMAD 10 detectsthe hidden platform 36, it may turn counter-clockwise 60 degrees andwait for 3 seconds, then turn clockwise for 60 degrees and wait 3seconds, then another 60 degree clockwise turn and 3 second wait, andfinally turn counter-clockwise returning to its original heading, allunder control of system 12. The simulation ends at this point, and thecurrent state of the simulated nervous system 12 is saved to hard disk.Otherwise, after 3 theta cycles, NOMAD 10 may choose a new heading basedon activity in the motor area (M_(HDG)) of the simulated nervous system12. From its original heading, NOMAD 10 may first turn counter-clockwise60 degrees and wait for 3 seconds, then turn clockwise for 60 degreesand wait 3 seconds, then another 60 degree clockwise turn and 3 secondwait, and finally turn counter-clockwise returning to its originalheading, all under control of system 12. The average activity of M_(HDG)is calculated during the wait periods. A softmax algorithm may be usedto create a probability distribution for choosing a new heading based onthe following equation:

${{p({newhdg})} = \frac{\exp( {40( \overset{\_}{M_{HDG}({newhdg})} )} )}{\sum\limits_{{h = {{hdg} - 60}},{hdg},{{hdg} + 60}}\;{\exp( {40( \overset{\_}{M_{HDG}(h)} )} )}}};$where newhdg is a possible new heading for NOMAD 10, M_(HDG)(newhdg) isthe average activity of M_(HDG) at a possible new heading, hdg is thecurrent heading, and h has three positions (current heading, currentheading less 60 degrees, current heading plus 60 degrees). After thesimulated nervous system 12 selects a new heading, NOMAD 10 would beoriented to the new heading and then proceed forward for another 3 thetacycles before activating the heading selection process again.Neuronal Units/Dynamics—Generally.

A neuronal unit within a neural area V1 Color, V1 Width etc. issimulated by a mean firing rate model. The state of each unit isdetermined by a mean firing rate variable (s). The mean firing ratevariable of each unit corresponds to the average activity or firing ratevariable of a group of roughly 100 neurons during a time period ofapproximately 200 milliseconds.

Synaptic connections between neural units, both within and betweenneuronal areas V1 Color, V1 Width, etc. are set to be eithervoltage-independent or voltage-dependent, and either plastic ornon-plastic. Voltage-independent connections provide synaptic input to apostsynaptic neuron regardless of postsynaptic state of the neuron.Voltage-dependent connections represent the contribution of receptortypes (e.g. NMDA receptors) that require postsynaptic depolarization tobe activated.

In other words, a presynaptic neuron will send a signal along its axonthrough a synapse to a postsynaptic neuron. The postsynaptic neuronreceives this signal and integrates it with other signals being receivedfrom other presynaptic neurons.

A voltage independent connection is such that if a presynaptic neuron isfiring at a high rate, then a post-synaptic neuron connected to it viathe synapse will fire at a high rate.

A voltage dependent connection is different. If the postsynaptic neuronis already firing at some rate when it receives a presynaptic inputsignal, then the voltage-dependent connection will cause thepostsynaptic neuron to fire more. Since the postsynaptic neuron isactive, i.e. already firing, this neuron is at some threshold level whenreceiving the input signal. Therefore, this presynaptic connection willmodulate the postsynaptic neuron to fire even more. Thevoltage-dependent connection, no matter how active the presynapticneuron is, would have no effect on the postsynaptic neuron if the latterwere not above the threshold value. Thus, the postsynaptic neuron has tohave some given threshold of activity to be responsive or modulated by avoltage-dependent synaptic connection.

Neuronal Units.

The mean firing rate (s) of each neuronal unit ranges continuously from0 (quiescent) to 1 (maximal firing). The state of a neuronal unit isupdated as a function of its current state and contributions fromvoltage-independent, and voltage-dependent inputs. Thevoltage-independent input to unit i from unit j is:A _(ij) ^(V1)(t)=c _(ij) s _(j)(t);where s_(j)(t) is the activity of unit j, and c_(i,j) is the connectionstrength from unit j to unit i. The voltage-independent postsynapticinfluence, POST_(i) ^(V1), on unit i is calculated by summing over allthe inputs onto unit i:

${{{POST}_{i}^{VI}(t)} = {{\varphi( {{POST}_{i}^{VI}( {t - 1} )} )} + {( {1 - \varphi} )( {\sum\limits_{l = 1}^{M}\;{\sum\limits_{j = 1}^{N_{l}}\;( {A_{ij}^{VI}(t)} )}} )}}};$where M is the number of different anatomically defined connection types(see Table 2), N₁ is the number of connections of type M projecting tounit i, and φ is the persistence of synaptic input.

The voltage-dependent input to unit i from unit j is:

A_(ij)^(VD)(t) = Φ(POST_(i)^(VI)(t))c_(ij)s_(j)(t), where${\Phi(x)} = \{ {\begin{matrix}{0;} & {x < \sigma_{i}^{vdep}} \\{x;} & {otherwise}\end{matrix};} $where σ_(i) ^(vdep) is a threshold for the postsynaptic activity belowwhich voltage-dependent connections have no effect (see Table 1).

The voltage-dependent postsynaptic influence on unit i, POST_(i) ^(VD),is given by:

${{{POST}_{i}^{VD}(t)} = {{\varphi( {{POST}_{i}^{VD}( {t - 1} )} )} + {( {1 - \varphi} )( {\sum\limits_{l = 1}^{M}\;{\sum\limits_{j = 1}^{N_{l}}\;( {A_{ij}^{VD}(t)} )}} )}}};$

A new activity, s_(i)(t+1), is chosen based on the sum of thepostsynaptic influences on neuronal unit i:

${{POST}_{i} = {{\sum\limits_{j = 1}^{N_{VI}}\;{POST}_{j}^{VI}} + {\sum\limits_{k = 1}^{N_{VD}}\;{POST}_{k}^{VD}}}};$

The new activity for the neuronal unit is the activity level at thenewly chosen phase, which is then subjected to the following activationfunction:

s_(i)(t + 1) = ϕ(tanh (g_(i)(POST_(i) + ω_(S_(i))(t)))), where${\phi(x)} = \{ {\begin{matrix}{0;} & {x < \sigma_{i}^{fire}} \\{x;} & {otherwise}\end{matrix};} $where ω determines the persistence of unit activity from one cycle tothe next, g_(i) a scaling factor, and σ_(i) ^(fire) is a unit specificfiring threshold.

Specific parameter values for neuronal units are given in Table 1:

TABLE 1 Parameter Values. Area Size σ-fire σ-vdep ω g V1 (8) 60 × 80 — —— — HD  1 × 360 — — — — R+ 1 × 1 — — — — R− 1 × 1 — — — — BF 1 × 1 — — —— V2/4-color (4) 6 × 8 0.20 0.10 0.0 1.0 V2/4-width (4) 15 × 20 0.200.10 0.0 1.0 IT 30 × 30 0.20 0.10 0.0 1.0 ITi 15 × 15 0.20 0.10 0.15 1.0PR 30 × 30 0.20 0.10 0.0 1.0 ATN 30 × 30 0.10 0.10 0.50 1.0 ATNi 30 × 300.10 0.10 0.15 1.0 M_(HDG)  1 × 60 0.0 0.10 0.0 1.0 M_(HDG)i  1 × 60 0.00.10 0.0 1.0 S 4 × 4 0.0 0.10 0.0 1.0 EC_(IN) 30 × 30 0.10 0.10 0.50 1.0EC_(IN)i 15 × 15 0.02 0.10 0.0 1.0 EC_(OUT) 30 × 30 0.10 0.10 0.50 1.0EC_(OUT)i 15 × 15 0.02 0.10 0.0 1.0 DG 30 × 30 0.10 0.10 0.50 0.75DG_(FB)i 15 × 15 0.02 0.10 0.0 1.0 CA3 15 × 15 0.05 0.10 0.50 0.75CA3_(FB)i 8 × 8 0.02 0.10 0.0 1.0 CA3_(FF)i 15 × 15 0.02 0.10 0.0 1.0CA1 20 × 20 0.05 0.10 0.50 0.75 CA1_(FB)i 10 × 10 0.02 0.10 0.0 1.0CA1_(FF)i 10 × 10 0.02 0.10 0.0 1.0 Specific values of parametersdefining properties of the neuronal units are given in Table 1.

As shown in Table 1, areas V1, HD, R+, R−, and BF are input areas andtheir activity is based on the camera image, odometry, and IR sensorsrespectively. Areas V1 and V2/V4 have 4 sub-areas for color (red, green,blue, and yellow) and 4 sub-areas for varying widths consistent with theexample of the enclosed environment 28 in which NOMAD 10 navigates.Table 1 indicates the number of neuronal units in each area or sub-area(Size). Neuronal units in each area have a specific firing threshold(σ-fire), and a threshold above which voltage-dependent connections canhave an effect (σ-vdep), a persistence parameter (ω), and a scalingfactor (g).

Synaptic connections for the neuronal units are given in Table 2:

TABLE 2 Properties of Anatomical Projections. Projection Arbor pc_(ij)(0) type φ η k1 k2 V1-color →V2/4-color [ ] 1 × 1 1.00 003, 0.050VI — 0.00 0.00 0.00 V2/4-color →V2/4-color(intra) [ ] 2 × 2 0.40 0.5,0.6 VD — 0.00 0.00 0.00 V2/4-color→V2/4-color(inter) [ ] 3 × 3 1.00−0.0012, −0.28 VI — 0.00 0.00 0.00 V1-width→V2/4-width [ ] 1 × 1, 2 × 2,3 × 3, 4 × 4 1.00 0.008, 0.009 VI — 0.00 0.00 0.00V2/4-width→V2/4-width(intra) [ ] 2 × 2 0.40 0.5, 0.6 VD — — — —V2/4-width →V2/4-width(inter) [ ] 3 × 3 1.00 −0.012, −0.014 VI — — — —V2/V4→IT non-topo 0.05 0.03, 0.04 VI — — — — IT→IT [ ] 1 × 1 1.00 0.08,0.14 VI — — — — IT→ITi Θ 2, 3 1.00 0.06, 0.08 VI — — — — ITi→IT [ ] 1 ×1 1.00 −0.36, −0.50 VI — — — — V2/V4→Pr [ ] 1 × 1 0.25 0.25, 0.30 VD — —— — Pr→Pr Θ 4, 6 1.00 −0.06, −0.08 VI — — — — HD→ATN [ ] 30 × 2 0.200.01, 0.02 VI — — — — ATN→ATNi Θ 10, 15 0.25 0.01, 0.02 VI — — — —ATNi→ATN [ ] 1 × 1 1.00 −0.36, −0.50 VI — — — — HD→M_(HDG) [ ] 1 × 11.00 0.01, 0.01 VI — — — — M_(HDG) → M_(HDG)i Θ 20, 30 0.50 0.10, 0.20VI — — — — M_(HDG)i→ M_(HDG) [ ] 1 × 1 1.00 −0.36, −0.50 VI — — — —ATN→EC_(IN) non-topo 0.001 0.40, 0.50 VI — — — — IT→EC_(IN) non-topo0.001 0.40, 0.50 VI — — — — Pr→EC_(IN) non-topo 0.001 0.40, 0.50 VI — —— — EC_(IN)→EC_(OUT) non-topo 0.05 0.04, 0.08 VI — — — —EC_(IN)→EC_(IN)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — EC_(IN)i→EC_(IN) [ ]1 × 1 1.00 −0.90, −1.20 VI — — — — EC_(IN)→DG [ ] 3 × 3 0.10 0.45, 0.60VI 0.75 0.05 0.90 0.45 EC_(IN)→CA3 [ ] 3 × 3 0.05 0.15, 0.20 VI 0.750.05 0.90 0.45 EC_(IN)→CA1 [ ] 3 × 3 0.04 0.30, 0.40 VI 0.75 0.05 0.900.45 EC_(OUT)→ATN non-topo 0.01 0.40, 0.45 VD — — — — EC_(OUT)→ITnon-topo 0.01 0.40, 0.45 VD — — — — EC_(OUT)→Pr non-topo 0.01 0.40, 0.45VD — — — — EC_(OUT)→EC_(IN) non-topo 0.05 0.04, 0.08 VI — — — —EC_(OUT)→EC_(OUT)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — EC_(OUT)i→EC_(OUT)[ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — DG→CA3 [ ] 3 × 3 0.03 0.45, 0.60VI — — — — DG→DG_(FB)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — DG_(FB)i→DG [ ]1 × 1 1.00 −0.90, −1.20 VI — — — — DG→CA3_(FF)i Θ 2, 3 0.10 0.45, 0.60VI — — — — CA3_(FF)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — CA3 →CA1[ ] 3 × 3 0.08 0.45, 0.60 VI 0.75 0.05 0.90 0.45 CA3 →CA3 non-topo 0.100.15, 0.20 VI 0.75 0.05 0.90 0.45 CA3 →CA3_(FB)i Θ 2, 3 0.10 0.45, 0.60VI — — — — CA3_(FB)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — CA3→CA1_(FF)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — CA1_(FF)i→CA1 [ ] 1 × 11.00 −0.90, −1.20 VI — — — — CA1 →EC_(OUT) [ ] 3 × 3 0.25 0.60, 0.75 VI0.75 0.05 0.90 0.45 CA1 →M_(HDG) # [ ] 3 × 3 1.00 0.01, 0.02 VD — 0.050.90 0.45 CA1 →S # [ ] 3 × 3 1.00 0.01, 0.02 VD —  0.005 0.90 0.45 R+ →Snon-topo 1.00 0.25, 0.25 VI — — — — R− →S non-topo 1.00 0.25, 0.25 VI —— — — BF → EC_(IN), EC_(OUT), DG, CA3, CA1 non-topo 0.05 −0.01, −0.02 VI— — — — Specific properties of anatomical projections and connectiontypes of the simulated nervous system are given in Table 2.

As indicated in Table 2, a presynaptic neuronal unit connects to apostsynaptic neuronal unit with a given probability (p) and givenprojection shape (Arbor). This arborization shape can be rectangular “[]” with a height and width (h×w), doughnut shaped “Θ” with the shapeconstrained by an inner and outer radius (r1, r2), or non-topographical“non-topo” where any pairs of presynaptic and postsynaptic neuronalunits have a given probability of being connected. The initialconnection strengths, c_(ij)(0), are set randomly within the range givenby a minimum and maximum value (min, max). A negative value forc_(ij)(0), indicates inhibitory connections. Connections marked with“intra” denote those within a visual sub-area and connections markedwith “inter” denote those between visual sub-areas. Projections marked #are value-dependent. As already mentioned, a connection type can bevoltage-independent (VI), or voltage-dependent (VD). φ denotes thepersistence of the connection. Non-zero values for θ, k₁, and k₂ signifyplastic connections.

Synaptic Plasticity—Value Independent Plasticity.

Synaptic strengths are subject to modification according to a synapticrule that depends on the phase and activities of the pre- andpostsynaptic neuronal units. Plastic synaptic connections are eithervalue-independent (see EC_(IN)→DG,CA3,CA1; DG→CA3; CA3→CA1; CA1→EC_(OUT)in FIG. 3B and Table 2) or value-dependent (see CA1→S, CA1→M_(HDG)).Both of these rules are based on a modified BCM learning rule in whichsynaptic change (Δc_(ij)) is a function of the post- and pre-synapticneuronal unit's activity and a sliding threshold (θ) as showngraphically in the insert next to FIG. 3B. Synapses between neuronalunits with strongly correlated firing phases are potentiated andsynapses between neuronal units with weakly correlated phases aredepressed; the magnitude of change is determined as well by pre- andpostsynaptic activities.

Thus, value-independent synaptic changes in c_(ij) are given by:Δc _(ij)(t+1)=ηs _(i)(t)BCM(s _(i))where s_(i)(t) and s_(j)(t) are activities of post- and presynapticunits, respectively, and η is a fixed learning rate. The function BCM isimplemented as a piecewise linear function, taking post-synapticactivity as input, which is defined by a sliding threshold, θ, twoinclinations (k₁, k₂) and a saturation parameter ρ (ρ=6 throughout):

${B\; C\;{M(s)}} = \{ \begin{matrix}{{{- k_{1}}s};} & {s \leq {\theta/2}} \\{{k_{1}( {s - \theta} )};} & {{\theta/2} < s \leq \theta} \\{{k_{2}{{\tanh( {\rho( {s - \theta} )} )}/\rho}};} & {otherwise}\end{matrix} $

The threshold is adjusted based on the post-synaptic activity:Δθ=0.25(s ²θ)

Value-independent plasticity is subject to weight normalization toprevent unbounded potentiation:

${c_{ij} = \frac{c_{ij}}{{sqrt}( {\sum\limits_{k = l}^{K}\; c_{kj}^{2}} )}};$where c_(ij) is a particular connection, and K is the total number ofconnections onto neuronal unit j.Synaptic Plasticity—Value Dependent Plasticity.

The rule for value-dependent plasticity differs from thevalue-independent rule in that synaptic change is governed by thepre-synaptic activity, post-synaptic activity, and temporal differenceTD from the value systems (see above re Value System and TemporalDifference Learning). The synaptic change for value-dependent synapticplasticity is given by:Δc _(ij)(t+1)=ηs _(i)(t)s _(j)(t)TD(t);where TD(t) is the temporal difference value at time t.Hidden Platform Task.

The hidden platform task assesses the BBD's spatial and episodic memoryand involves both a training trial phase and a probe trial phase. NOMAD10 begins a training trial from each of four starting locations on thefloor 32 of enclosed environment 28 (see FIGS. 2A-2B) and explores theenclosure until it encounters the hidden platform 36 or it times out(after 1000 seconds or 5000 cycles). A training block is defined as aset of four such trials from each of the four starting locations. Fourtraining blocks (16 total trials) are completed by the BBD duringtraining, and the results are described below in connection with FIGS.4-5 After training is completed, the BBD undergoes a probe trial, inwhich the hidden platform 36 is removed, to assess the BBD's memoryperformance, which is explained below in relation to FIG. 6.

Training and probing in the hidden platform tasks are repeated with nineslightly different BBD “subjects”. Each “subject” included the samephysical device, i.e. NOMAD 10, but each possessed an altered simulatednervous system 12. This variability among “subjects” was a consequenceof random initialization in both the microscopic details of connectivitybetween individual neuronal units and the initial connection strengthsbetween those units. The overall connectivity among neuronal unitsremained similar among different “subjects”, however, inasmuch as thatconnectivity was constrained by the synaptic pathways, arborizationpatterns, and ranges of initial connection strengths (see FIGS. 3A-3Band Table 2 for specifics).

FIG. 4 shows representative trajectories of NOMAD 10 during 16respective hidden platform training trials. The position of NOMAD 10 wasrecorded by the overhead cameras (not shown) above the enclosures 28.Each box in FIG. 4 shows a trajectory of NOMAD 10 during a trial(T1-T16), and the number of simulation cycles it took for NOMAD 10 tofind the hidden platform 36. NOMAD 10 started its trajectory from thesmall black square shown in each figure and moved to find the hiddenplatform 36. The hidden platform 36 is shown as a green circle in thefigure if NOMAD 10 found the platform 36 within 5000 simulation cyclesand is shown as a red circle if it did not find the hidden platform 36during the trial period. By the midpoint of its training, i.e. aftertrial T8, NOMAD 10 is able to make trajectories essentially directlytoward the hidden platform 36 from its starting point.

During each such simulation cycle of the BBD, sensory input to thesimulated nervous system 12 is processed, the states of all neuronalunits are computed, the connection strengths of all plastic connectionsare determined, and motor output is generated. In the trials, executionof each simulation cycle required approximately 200 milliseconds of realtime. Every simulation cycle, the positions of NOMAD 10, its heading,and the state of all neuronal units are recorded and saved on a harddisk.

Results. Nine “subjects” were run on the hidden platform task.

Overall, the group of “subjects” learned the hidden platform task andshowed improvement, as measured by the time to find the hidden platform36, as training progressed. This improvement is indicated in FIG. 5,which is a graph of the mean escape times of the nine “subjects” testedduring the hidden platform task. As shown in FIG. 5, each “subject” ranfour trial blocks with four trials within each block. Error bars denotethe standard deviation. Escape times were significantly shorter forblocks 3 and 4 than block 1 (p<0.01) and than block 2 (p<0.05). TheP-values are derived from the known Wilcoxon sign rank test.

Probe Trial. During the probe trial, in which the hidden platform 36 wasremoved and “subjects” explored the environment 28 for 5000 simulationcycles or approximately 17 minutes, “subjects” spent a significantamount of time searching in the region where the hidden platform 36would have been located. Seven “subjects” were tested in the probe trialand the “subjects” spent approximately half their time (μ=0.50, σ²=0.23)in the quadrant of the enclosure 36 that had contained the hiddenplatform 36.

FIGS. 6A-6B show trajectories of “subjects” during the probe trial inwhich the hidden platform 36 was removed. The red circles denote thelocation of the hidden platform 36 during the training trials. FIG. 6Ashows a “subject” with average performance spent 46% of its time in thequadrant where the hidden platform 36 was used during the trainingtrials described above. FIG. 6B shows that the “subject” thatrepresented a better performance during the probe trial spent 85% of thetime in the hidden platform quadrant.

Neural Response. Many of the neuronal units in the hippocampal areas ofthe simulated nervous system 12 have responses typical of what arecalled hippocampal “place” cells where the neuronal unit was activeexclusively while NOMAD 10 was in a specific region of the environment28. This is indicated in FIGS. 7A-7D, which show four representative“place” cells recorded in simulated neuronal unit CA1 of thehippocampus. Each rectangle represents the enclosure 28 that NOMAD 10explores and each pixel represents a one foot square in the enclosure28. The color of the pixel represents the activity of a given CA1neuronal unit and is normalized from quiescent (white) to maximal firingrate (black). The red circle for these figures denotes the location ofthe hidden platform 36.

The response of neuronal units is not only place specific, but alsocontext specific. This is a critical requirement for an episodic memorysystem and could only happen in a system that has architecturefacilitating the integration of inputs over time. Many of the neuronalunits respond at a particular place in the enclosure 28 depending onNOMAD's trajectory. These units are called “journey dependent cells”.Units have been found with retrospective coding (i.e. current activityis based on its prior experience to that point) by looking at cells inall the simulated hippocampus areas that responded near the hiddenplatform 36 (e.g., within 3 feet). In all training and probe trials,NOMAD 10 visited this location. However, as shown in FIGS. 8A-8F, themajority of these cells responded only on a subset of trials, while asmaller proportion of units responded on a majority of the trialsregardless of trajectory.

More specifically, FIGS. 8A-8F are charts showing journey dependent andjourney independent cells in the hippocampus model. Neuronal units thatwere active (average activity >0.1) and that had “place” fields centeredwithin 3 feet of the hidden platform 36 were used in the analysis.Journey independent cells were active in 5 or more trials, moderatelyjourney dependent cells were active in 3 to 4 trials, and highly journeydependent cells were active in only 1 to 2 trials. FIGS. 8A-8F show theproportions of journey independent and journey dependent cells in eachhippocampal subregion (EC_(IN), EC_(OUT), DG, CA3, CA1) and in thehippocampus overall (FIG. 8F).

In many cases NOMAD 10 demonstrated stereotypical trajectories from agiven starting position to the hidden platform 36. Roughly 10% of thehighly journey dependent cells were active near the location of thehidden platform 36 with similar trajectories (i.e. trials 9 and 13,trials 10 and 14, trials 11 and 15, trials 12 and 16).

FIGS. 9A-9H illustrate a representative “journey dependent” “place” cellin the neuronal area CA1. This cell was most active near the hiddenplatform 36 (open circle in the figures) on training trials 9 and 13where NOMAD 10 started from the wall 30 at the top of the figures.Activity is normalized to the maximum firing rate over all trainingtrials for this cell where white denotes no activity and black denotesmaximal activity.

FIG. 10 is an exemplary illustration of a computer system in accordancewith various embodiments of the invention. Although this diagram depictscomponents as logically separate, such depiction is merely forillustrative purposes. It will be apparent to those skilled in the artthat the components portrayed in this figure can be arbitrarily combinedor divided into separate software, firmware and/or hardware components.Furthermore, it will also be apparent to those skilled in the art thatsuch components, regardless of how they are combined or divided, canexecute on the same computing device or can be distributed amongdifferent computing devices connected by one or more networks or othersuitable communication means.

In various embodiments, the components illustrated in FIG. 10 can beimplemented in one or more programming languages (e.g., C, C++, Java™,and other suitable languages). Components can communicate using MessagePassing Interface (MPI) or other suitable communication means, includingbut not limited to shared memory, distributed objects and Simple ObjectAccess Protocol (SOAP). MPI is an industry standard protocol forcommunicating information between computing devices (or nodes). In oneembodiment, the system can be deployed on a multi-processor computerarchitecture such as (but not limited to) a Beowulf cluster. Beowulfclusters are typically comprised of commodity hardware components (e.g.,personal computers running the Linux operating system) connected viaEthernet or some other network. The present disclosure is not limited toany particular type of parallel computing architecture. Many other sucharchitectures are possible and fully within the scope and spirit of thepresent disclosure.

Referring to FIG. 10, master component 1302 can coordinate theactivities of the other components according to commands received fromclient 1304. In one embodiment, the client 1304 can be a stand-aloneprocess or that programmatically controls the master according to ascript or other scenario and/or in reaction to client information (e.g.,neural activity, sensor readings and camera input) received from themaster. Client commands can instruct the master 1302 to start or stopthe brain-based device BBD experiment, save the experiment state on datastore 1312, read the experiment state from the data store 1312, set therunning time/cycles in which the experiment will execute, and setparameters of the neural simulators 1310.

In another embodiment, the client 1304 can be a user interface thatreceives information from the master 1302 and allows a user tointeractively control the system. By way of a non-limiting example, auser interface can include one or more of the following: 1) a graphicaluser interface (GUI) (e.g., rendered with Hypertext Markup Language); 2)an ability to respond to sounds and/or voice commands; 3) an ability torespond to input from a remote control device (e.g., a cellulartelephone, a PDA, or other suitable remote control); 4) an ability torespond to gestures (e.g., facial and otherwise); 5) an ability torespond to commands from a process on the same or another computingdevice; and 6) an ability to respond to input from a computer mouseand/or keyboard. This disclosure is not limited to any particular GUI.Those of skill in the art will recognize that many other user interfacesare possible and fully within the scope and spirit of this disclosure.

The neuronal units for each neural area are each assigned to a neuralsimulator 1310. Each neural simulator 1310 is responsible forcalculating the activity of the neuronal units that have been assignedto it. A given neural area's neuronal units may be distributed acrossone or more neural simulators 1310. In various embodiments, there can beone neural simulator 1310 per Beowulf node. In order to optimizeperformance, neuronal units can be distributed among neural simulatorssuch that the average number of synaptic connections on the neuralsimulators is approximately the same. In other embodiments, neuronalunits can be distributed such that the average number of neuronal unitsper neural simulator is approximately the same. Neural simulatorsperiodically or continuously exchange the results of calculating theactivity of their neuronal units with other neural simulators and themaster. This information is required so that neuronal units on otherneural simulators have up-to-date pre-synaptic inputs. The masterprovides actuator commands to NOMAD 10 based on the neural activityreceived from the neural simulators 1310.

The master periodically receives image data from image grabber 1306 anddistributes it to the neural simulators 1310 and to the client 1304. Inone embodiment, the images are taken from the CCD camera 16 mounted onNOMAD 10 that sends the RGB video images, via an RF transmitter, to anImageNation PXC200 frame grabber 1306. The image is then spatiallyaveraged to produce a pixel image. Gabor filters can be used to detectedges of vertical and horizontal orientations (as briefly describedabove). The output of the Gabor function is mapped directly onto theneuronal units of the corresponding V1 width neural areas. Color filtersare also applied to the image, with the outputs of the color filtersbeing mapped directly onto the neuronal units of V1 Color.

The master 1302 also periodically acquires sensor data from NOMAD 10component 1308 and distributes it to the neural simulators 1310. In oneembodiment, a micro controller (PIC17C756A) onboard the NOMAD 10 samplesinput and status from its sensors and controls an RS-232 communicationbetween the NOMAD 10 base and master 1302. Sensor information caninclude, in addition to video information previously described, gripperstate, camera position, infrared detectors, whisker deflection, wheelspeed and direction and odometer count.

FIG. 11 is a flow diagram illustration of neural simulatorinitialization in accordance with various embodiments of the invention.Although this figure depicts functional steps in a particular order forpurposes of illustration, the process is not necessarily limited to anyparticular order or arrangement of steps. One skilled in the art willappreciate that the various steps portrayed in this figure can beomitted, rearranged, performed in parallel, combined and/or adapted invarious ways. In step 1402, it is determined based on command(s) fromthe client 1304 whether or not a saved trial should be retrieved fromthe data store 1312 or whether a new trial should be started. If thetrial is to be retrieved from the data store, this is performed in step1410. In various embodiments, the trial state can be stored as anExtensible Markup Language (XML) document, a plain text file, or abinary file. Otherwise, in step 1404 neuronal units are createdaccording to the parameters given in Table 1. Next, in step 1406synaptic connections are created between the neuronal units according tothe parameters in Table 2. Finally, each neuronal unit is assigned to aneural simulator in step 1408.

FIG. 12 is a flow diagram illustration of the master component inaccordance with various embodiments of the invention. Although thisfigure depicts functional steps in a particular order for purposes ofillustration, the process is not necessarily limited to any particularorder or arrangement of steps. One skilled in the art will appreciatethat the various steps portrayed in this figure can be omitted,rearranged, performed in parallel, combined and/or adapted in variousways.

In step 1502 the master 1302 broadcasts image and sensor data that ithas acquired from the image grabber and NOMAD 10 to the neuralsimulators and the client. In step 1504, the master broadcasts anycommands it may have received to the neural simulators. In step 1506, itis determined whether or not the client 1304 has directed the master1302 to quit the experiment. If so, the master ceases the experiment(which may include saving the state of the experiment to the datastore). Otherwise, in step 1508 the updated information is provided tothe client which could serve to update a GUI. In step 1510, neuronalunit activity from the neural simulators 1310 is shared among allcomponents (e.g., via MPI). The neuronal activity can be provided insome form to the client as part of the client information. Finally, itis determined whether or not there are any remaining cycles left in thesimulation. If not, the trial terminates. Otherwise, the master 1302returns to step 1502.

FIG. 13 is a flow diagram illustration of a neural simulator inaccordance to various embodiments of the invention. Although this figuredepicts functional steps in a particular order for purposes ofillustration, the process is not necessarily limited to any particularorder or arrangement of steps. One skilled in the art will appreciatethat the various steps portrayed in this figure can be omitted,rearranged, performed in parallel, combined and/or adapted in variousways.

In step 1602, the neural simulators 1310 accept image and sensor datathat is broadcast by the master 1302. In step 1604, client commandsbroadcast by the master 1302 are accepted. In step 1606, it isdetermined whether or not the client 1304 has directed the master 1302to quit the trial. If so, the neural simulators 1310 complete theirexecution. Otherwise, in step 1608 the value of the neuronal unitsassigned to the neural simulators are calculated. In step 1610, thestrengths of plastic connections are calculated. Local neuronal unitactivity is shared in step 1612 with other neural simulators and themaster. In addition, neuronal activity from other neural simulators isacquired and used to refresh local values. Finally, it is determined instep 1614 whether or not there are any remaining cycles left in thetrial. If not, the trial terminates. Otherwise, the neural simulators1310 return to step 1602.

Various embodiments may be implemented using a conventional generalpurpose or a specialized digital computer or microprocessor(s)programmed according to the teachings of the present disclosure, as willbe apparent to those skilled in the computer art. Appropriate softwarecoding can readily be prepared by skilled programmers based on theteachings of the present disclosure, as will be apparent to thoseskilled in the software art. The invention may also be implemented bythe preparation of integrated circuits or by interconnecting anappropriate network of conventional component circuits, as will bereadily apparent to those skilled in the art.

Various embodiments include a computer program product which is astorage medium (media) having instructions stored thereon/in which canbe used to program a general purpose or specialized computingprocessor/device to perform any of the features presented herein. Thestorage medium can include, but is not limited to, one or more of thefollowing: any type of physical media including floppy disks, opticaldiscs, DVDs, CD-ROMs, microdrives, magneto-optical disks, ROMs, RAMs,EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or opticalcards, nanosystems (including molecular memory ICs); and any type ofmedia or device suitable for storing instructions and/or data. Variousembodiments include a computer program product that can be transmittedover one or more public and/or private networks wherein the transmissionincludes instructions which can be used to program a computing device toperform any of the features presented herein.

Stored in one or more of the computer readable medium (media), thepresent disclosure includes software for controlling both the hardwareof the general purpose/specialized computer or microprocessor, and forenabling the computer or microprocessor to interact with a human user orother mechanism utilizing the results of the present invention. Suchsoftware may include, but is not limited to, device drivers, operatingsystems, execution environments/containers, and applications.

Summary

The BBD of the present invention demonstrates the acquisition and recallof spatial memories by observing the hippocampus unit activity duringits behavior in the environment 28, and is shown to have contextdependent neural responses, which is a prerequisite for episodic memory.

The BBD methodology described above is used for investigating episodicand spatial memory. Unlike prior computational models of thehippocampus, the BBD is employed in the environment 28 and nopresumptions about the environmental inputs are made, nor areassumptions made about the behavioral actions of NOMAD 10 to solve thespatial memory task. Moreover, unlike prior robotic systems that havesome abstraction of the hippocampus to build spatial memories, the BBDhas neural dynamics coupled with detailed neuroanatomy (see Table 1 andTable 2) at a systems neuroscience level.

The BBD model described herein takes into consideration the macro andmicro anatomy between the hippocampus and cortex, as well as theneuronal areas within the hippocampus. The BBD developed mappings fromthe neural responses in the hippocampus to purposeful behavior (see FIG.4-FIG. 6). It developed “place” cells, comparable to those seen in arodent, without assumptions or instructions (see FIGS. 7 and 9). Theseresponses came about based on autonomous exploration of NOMAD 10 and theintegration of three input streams (visual what or IT, visual where orPr, and self-movement or ATN) over time.

It is believed that the multiple loops from the cortex to hippocampus(FIG. 3A) and within the hippocampus proper (FIG. 3B) are important forintegrating inputs over time and building spatial memory that depends oncontext (see FIGS. 8-9). These context-dependent responses are criticalfor the acquisition and recall of sequences of multimodal memories andare the hallmark of episodic memory.

1. A mobile brain-based device for navigating in a real-worldenvironment, comprising: a) a mobile adaptive device having sensors forsensing multi-modal information in the real-world environment; b) asimulated nervous system for receiving and processing the multi-modalinformation sensed by the sensors and, in response, for outputtinginformation to control movement of said mobile adaptive device in theenvironment; and c) wherein said simulated nervous system exhibits themorphology and functionality of a part of a human brain, and includessensor input streams, an output motor stream, and a neural areaanalogous to the hippocampus of the human brain being coupled to saidsensor input streams and to said output motor stream of the simulatednervous system.
 2. A mobile brain-based device according to claim 1,wherein said sensor input streams include a neuronal area analogous tothe cortex of the human brain.